Op-cbio130549 1..7

Motivation: Our goal is to develop a screening platform for quantitative profiling of colony organizations in 3D cell culture models. The 3D cell culture models, which are also imaged in 3D, are functional assays that mimic the in vivo characteristics of the tissue architecture more faithfully than the 2D cultures. However, they also introduce significant computational challenges, with the main barriers being the effects of growth conditions, fixations and inherent complexities in segmentation that need to be resolved in the 3D volume. Results: A segmentation strategy has been developed to delineate each nucleus in a colony that overcomes (i) the effects of growth conditions, (ii) variations in chromatin distribution and (iii) ambiguities formed by perceptual boundaries from adjacent nuclei. The strategy uses a cascade of geometric filters that are insensitive to spatial nonuniformity and partitions a clump of nuclei based on the grouping of points of maximum curvature at the interface of two neighboring nuclei. These points of maximum curvature are clustered together based on their coplanarity and proximity to define dissecting planes that separate the touching nuclei. The proposed curvature-based partitioning method is validated with both synthetic and real data, and is shown to have a superior performance against previous techniques. Validation and sensitivity analysis are coupled with the experimental design that includes a non-transformed cell line and three tumorigenic cell lines, which covers a wide range of phenotypic diversity in breast cancer. Colony profiling, derived from nuclear segmentation, reveals distinct indices for the morphogenesis of each cell line. Availability: All software are developed in ITK/VTK and are available at https://vision.lbl.gov/Software/3DMorphometry. Contact: [email protected] or [email protected] Supplementary information: Supplementary data are available at Bioinformatics online. Received on April 26, 2013; revised on August 16, 2013; accepted on